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Complex Electronics (CE) are now programmed to perform tasks that were previously 
handled in software, such as communication protocols. Many of the methods used to 
develop software bare a close resemblance to CE development. For instance, Field 
Programmable Gate Arrays (FPGAs) can have over a million logic gates while system-on- 
chip (SOC) devices can combine a microprocessor, input and output channels, and 
sometimes an FPGA for programmability. With this increased intricacy, the possibility of 
“software-like” bugs such as incorrect design, logic, and unexpected interactions within the 
logic is great. 

Since CE devices are obscuring the hardware/software boundary, we propose that mature 
software methodologies may be utilized with slight modifications in the development of 
these devices. Software Process Assurance for Complex Electronics (SPACE) is a 
research project that looks at using standardized S/W Assurance/Engineering practices to 
provide an assurance framework for development activities. Tools such as checklists, best 
practices and techniques can be used to detect missing requirements and “bugs” earlier in 
the development cycle creating a development process for CE that will be more easily 
maintained, consistent and configurable based on the device used. 
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The Quandary 


■ICOrpplex 0ecTt:o^cs fGE) 
devices can have over 
one million gates and 
even a built in 
microprocessor. These 
devices are replacing 
conventional software in 
" majiy critical applications. 
How do we assure quality 
-on Hese devices? 
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How the Design Process For Complex 

Electronics should flow 
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Simple or Complex? 


The Federal Aviation Administration (FAA) provides a definition in 
DO-254, “Design Assurance Guidance for Airborne Electronic 
Hardware” document. It states “A hardware item is identified as 
simple only if a comprehensive combination of deterministic tests 
and analyses appropriate to the design assurance level can ensure 
correct functional performance under all foreseeable operating 
conditions with no anomalous behavior. When an item cannot be 
classified as simple , it should be classified as complex. An item 
constructed entirely from simple items may itself be complex.” 

Firmware is not CE. The most common definition is found in IEEE 
610.12-1990: “The combination of hardware device and computer 
instructions and data that reside as read-only software on that 
device.” 
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2. The complex electro. 

3. The design 
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•onics exdcutes.safety-critical functions 


lies executes mission-critical functions and is a single point of failure 
id to be highly complex 

;d to have significant risk due to one or more of these factors: 



sign is expected to have significant risk due to one or more of these factors: 

:auir ements 

_„ncems^vith the chosen-technology, suph as power consumption, design size 
for the chip, timing, packaging, or operating frequency 

TT: ~Hy innovative and untried design approach 
ly aggressive schedule 
Inexperience of the design team 


The complex electronics executes mission-critical functions but there is redundancy in the 
system 

The design is expected to be moderately complex 
3. The design is expected to have moderate risk due to one or more of these factors: 

i. Some requirements undefined or unstable 

ii. Somewhat innovative and untried design approach 

iii. Aggressive schedule 

iv. Design team contains some inexperienced members 


The complex electronics is classified as Low if it does not fall into either of the above categories 
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How do You Start the Process? 


■ Create an Assurance 
Plan for your device 

□ Can be stand alone or 
part of the larger 
assurance plan 

□ Plan is based on 
criticality of device(s) 

□ Get concurrence with the 
plan 



One plan can cover all 
devices needed 
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Assurance Planning is Very Important 
Assurance Planning Document 




ID Process Step 


Entrance Criteria are met. 


Concur with complex electronics criticality classification (high, moderate, or 
low). Include in the CEAP (below) 


Generate tailored Complex Electronics Assurance Plan (CEAP). This 
includes steps 4 through 14 below. 


Specify purpose and scope of plan. 


Determine Applicable and Reference standards and documents. 


Depie management of the assurance activities, including organization 
structure, roles and responsibilities, resources, schedule, reporting. 


Identify training and tools required for the assurance tasks. 


Specify tasks for Requirements phase, including reviews, audits, and 
analyses. 


1 3 Specify criteria and tasks for acceptance of complex electronics. 


20 Exit Criteria are met. 


Completion QA 
Date 
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Supporting Processes 


■ Controlling and monitoring the process used 
is very important. 


□ Configuration Management 

□ Audits 



□ Selecting the development process 

□ Problem reporting / monitoring 
Risk Management 
Ad Hoc is NOT a process! 
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CE Process Assurance 


Responsibility 


efine the system requirements 

Decompose system requirements down to sub-system 
level 

• Define system-level testing 


* Derive requirements for the .board or chip level 
^-Design electronics to m Settle requirements, using ' 
good engineering practices 

* Implerlient4he design p hardware 

* Test the hardware; Implement changes 


♦Identify If complex electronics can cause a hazard or 
are part of a hazard control 

* Ensure Inat design errors in complex electronics are 
considered as a failure mode 


* Assess entrance and exit criteria for each life cycle 
phase 

* Ensure traceability of the requirements through all levels 
of development 

* Analyze the products produced (documents, designs, 
etc.) against the requirements and the output of the 
previous phase 

* Perform white box analysis on CE designs and tests 
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The first step in any design process should be to define and 
document the requirements end constraints under which the CE 
must operate^This allows you to think through the issues and 
document any design decisions and trade-offs . 

Complex JSectronics desigrf is often 1 started based on the engineers 
||nowledge of the system, not defiled fequirejnents.HBP^ ^ 7 * 

Requirements Reviews 

■ Clear 

■ Concise 

■ Confirmable 

■ Traceable 

frterface Control Docurrrent verifications 
Signals pst 
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Requirements Review Checklist Snippet 


ID 

Topic 

Criteria 

Yes/No/ 

NA 

Comment 

2 

General 

Are the overall system configurations and operating modes 
defined? 



11 

Interfaces 

Is the data size and bit order defined? 



12 

Signals 

For each input signal, is the range of valid data defined? 



13 

Signals 

For each output signal, is the range of valid data defined? Is a 
typical value defined? 



20 

Initial 

Conditions 

During power-up or reset, do any lines or signals float? 



27 

Error 

Handling 

Is the behavior of the CE in response to an external failure or 
fault specified? 



28 

Timing 

Has the timing of critical signals been defined? 
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Design 

One major difference between CE and software 
is the aspect of timing and concurrency. 

□ Design reviews are important. 

□ Independent engineer should review the design 

□ Confirm the design supports the requirements 

□ Use a coding standard 

□ Have and follow Best Practices 
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FaifPT ree Ppaly s i s 
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Code P llpflementation 


Although HDlJis not true code, it shares many of 
Hie s^etea^ces^d^attri^Erte^^of software. 
Differences include: 



he placement of the logic blocks within the 
chip, and the routing between blocks, are 
some of the processes that occur during 
implementation (link). 
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Ease of Coding 


■ Coding Standards, Code Reviews and Best 
Practices work well on HDLs. They allow: 

□ Readability 

■ Standard Signal names 

■ Names do not change across boundaries 
Common register names 

□ Maintainability 

□ Common naming conventions 
Code reviews 

Etc.... 
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Code Example 


.Ucrarv 

T3E ^ 5 =..==. - 3 td logic 116^ . aTT; 

[ENT mi rsg^ 3 3 FORT ( 

clk: f we: ZZtT 3 td_iogic; 

rdata: IN 3 td logicj/ector (7 D-CWNT G O) 

AesI, EbsI : ITT 3 1 d_Io g ic_vs ct or (1 DOWHTO O) ; 

iAouLi=. r Bonn. : OUT logic vector (7 DDi#TTD O) j ; 

I TUT: o^ecrE ; 


;tue 


rXTT 
rst. : 


FRO! 


OF 


( c^k:. 


:dat( 


■) 


tyf: 


IRiVLT (O TO 3) OF s t d_Io g ic_ve ct or (7 DOWNTO O) ; 
:ey(7 DOWHTO O i ; 


rZTI 


: F c Ik p EVENT .AMD cik='0 
X F l we = p 1 r ]i T H EM 


TEEN 


Az=- ^ A3sl. 

3 



WHEN 

M OO n => 

re^ ( Q) 

: =rdata ; 

WHEN 

r, o: n => 

ieg C —) 

: =rdata ; 

WHEN 

r, :o n => 

( 2 ) 

: =rdata „■ 

WHEN 

OTHERS = 

> rec (3) : = 07d.au. a; 

END- CA5E; 




CASE iA3 s i_ 

TS 



WHEN 

M OO n => 

Aoizl=.<= 

-se (O) ; 

WHEN 

M Ol ri => 

A.OU t=- <= 

-sc- ( 1 . ) ; 

WHEN 

r TO n => 

Aoui t=- <= 

-egr (2 ) ; 

WHEN 

OTHE ER3 = 

> AOTJLt 

< = zsg i_ 3 j .r 

END- CASH 

! ; 



CASE Bse 

r\_ 75 



WHEN 

r. OO ri => 

3 oi JL H- <= 

-sc- (O) ; 

WHEN 

" Ol n => 

Bofut.<= 

-ecr ; 

WHEN 

TO n => 

BOTJLt< = 

-“O' (2 ) ; 

WHEN 

OTHE ES = 

> Bo\zt=-<=07&^ (3) ; 


CMC- 


TO 


END 

CMD FRGCE; 


[>TD one: 
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Example of Code Review Best Practices 


ID 

Topic 

Criteria 

Yes/No/ 

NA 

Comment 

5 

General 

All states in a state machine are defined or invalid states are 
returned to a known state 



25 

Clocks 

Do not generate the clock using combinational logic 



27 

Signals 

Names shall be self-explanatory 



34 

Reset 

Provides an Asynchronous reset line 



36 

Interfaces 

All asynchronous inputs are first synchronized before use 



39 

Functions 

The sensitivity list contains only the signals that should cause the 
process to be executed 



41 

Other 

Are unused functions within the IP cores identified? Is the accidental 
execution of these functions prevented? 
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Test 


While complex electronics use test benches 
and timing models, the idea of a well defined 
suite of test cases is common in both 
disciplines. This includes test plans, fault 
injection and error handling testing and 
verification. 





ih rp> ■-! •. Pih-" : 
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Best Practices 
Test Plans 

> Tracing to requirements 

> Feasible 

. Co^tJmoreiiarT’just success 
■ Fault Injection 
^ T es? VerifcalSon JP 

Proflem Tfend Anatysis / Tracking 
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Test Plan Review Checklist 


ID 

Topic 

Criteria 

Yes/No/NA 

Comment 

4 

General 

Is the functionality of the complex electronics (CE) in off- 
nominal mode being tested? 



14 

Interfaces 

Interfaces with COTS IP modules tested 



17 

Signals 

For each input signal, is the range of valid data tested? 



25 

Initial 

Conditions 

Is the state of programmable memory elements upon power- 
up and after a reset tested? 



27 

Error Handling 

Have all expected errors or faults been tested to verify they are 
handled correctly? 



30 

Tinning 

Has the timing of critical signals been tested? 



32 

Power/ 

Electrical 

Is the maximum allowable power and current to the CE being 
verified? 
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Reality Check 

■ Many assurance engineers, regardless of their 
specialty, have little understanding of the 
complexities of these devices. Any review done will 
only be to the level of knowledge of the assurance 
engineer. 

■ Three courses have been developed 

□ Introduction to CE 

□ CE Life Cycle 

□ Assurance of CE devices 

Techniques and checklists were created to ensure 
quality. 

Not all techniques/checklists used on every part. 
Dependent on criticality and project direction. 
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Process Flow Example 


CERP 


CEDP 


CEDDP 


IMPL 


V &, V 


Acce pta nee 
and Release 


OPS 



■Software- ■Qualify to Hardware Quality 
Transition 

Bi-diractianal Rsqu ii amants »racaabi5ity - 
Software Qualify, Safety and Reliability 
nnmnnnnnT TfreNno - Hardware Quality 
TP iS ■' .Xe^t B I gji ■' .Te^t F; munrese review - 

Sallware Quality. Hale ly and Reliability 

Systam Int&aratlQn Testing - Hardware 
Quality 
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How Complex Electronics Fits into 

Review Cycles 


SRR 


CE Design Process Flow 
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y D ec i s i o n ; Tif W^Ttees 

* Design Evaluation 

* Design Review 


* Failure Mode and Effect Analysis 

* Fault Tree Analysis 

*'gjjnctic5n anjJ Physical Confgurdtion Audits 

* Interface Ifnalysis 

* IRequ ire rife nts ^valuation 

* Meqdir^rnents Review 
■ Risk Analysis 
■Pfraceability Analysis 
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Rational for Change 
Effects' oh Internal Interfaces- 
Effects on External SnfleffaceS 
Effects on Hazards 


Effects on Operations 
Potential for introducing new bugs 
impact* of Change (Minor/Major and why) 
Tesfing/Velifications needed 
Things to Consider 
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Checklists 


■ Planning Phase 

■ Requirements Phase 

■ Preliminary Design Phase 

■ Detailed Design Phase 

■ Implementation Phase 

■ Testing Phase 

■ Operations Phase 

■ Assurance Planning 

■ Modifications or Upgrades 

■ Audits (Functional Configuration, Physical Configuration and 
In-Process) 

j Best Practices (Code Review) 
j Testing (Document Review) 
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Requirements Phase Process Checklist 

■ Overview 

■ Entrance Criteria 

■ Responsible Personnel 

■ Process Step - Lists Analysis to use/update, 
documentation(to create, update, etc.), 
supporting processes needed 

Exit Criteria 

Documentation Status 
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Conclusion 

■ Thanks to the NASA Software Assurance 
Research Program which funded this 
Research 

■ Contact I nformation 

- Richard Plastow 

- 21000 Brookpark Road, MS 50-4 

- Cleveland, OH 44135 

- Richard. A. PlastowccDnasa.gov 

- (216) 433-3431 
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